Evaluating Explainability Methods Intended for Multiple Stakeholders

Abstract Explanation mechanisms for intelligent systems are typically designed to respond specific user needs, yet in practice these tend have a wide variety of users. This can present challenge organisations looking satisfy the explanation needs different groups using an individual system. In this paper we explainability framework formed catalogue methods, and integrate with range projects within telecommunications organisation. Explainability methods split into low-level explanations high-level increasing levels contextual support their explanations. We motivate case-study explaining conclusions field network engineering experts non-technical planning staff evaluate our results feedback from two distinct groups; domain-expert telecommunication engineers non-expert desk agent staff. also investigate metrics model quality - Meet-In-The-Middle (MITM) Trust-Your-Neighbours (TYN). Our analysis offers new insights use similarity knowledge evaluation